A Large-Scale Gaussian Belief Propagation Solver for Kernel Ridge Regression

We introduce an efficient parallel implementation of a Kernel Ridge Regression solver, based on the Gaussian Belief Propagation algorithm (GaBP). Our approach can be easily used in Peer-to-Peer and grid environments, where there is no central authority that allocates work. Empirically, our solver has high accuracy in solving classification problems. We have tested our distributed implementation on large scale datasets using up to 1,024 CPUs of an IBM Blue Gene supercomputer. 1. KERNEL RIDGE REGRESSION Kernel Ridge Regression (KRR) implements a regularized form of the least squares method useful for both regression and classification. The non-linear version of KRR is similar to the SupportVector Machine (SVM) problem. However, in the latter, special emphasis is given to points close to the decision boundary, which is not provided by the cost function used by KRR.